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Abstract — This  paper  considers  cost  modeling 
efforts  within  the  Department  of  Defense  over  the  last 
30  years  to  formulate  an  approach  for  integrating 
cost  modeling  capability  with  the  systems  engineering 
tools  developed  under  the  Engineering  of  Complex 
Systems  Block  Program. 

I.  INTRODUCTION 

The  need  to  build  complex  warfare  systems  within 
an  industry  characterized  by  the  declining  defense  budget 
has  increased  the  role  of  cost  modeling  for  defense 
systems.  Accordingly,  systems  engineering  design 
synthesis  methodologies  must  incorporate  cost  modeling 
capabilities  to  insure  robust  tradeoff  analyses. 

Ideally,  a  sound  systems  engineering  design 
synthesis  methodology  assesses  the  tradeoff  between 
development  cost,  long  term  life  cycle  cost  (excluding 
development  cost),  time,  and  functional  performance. 
Figure  1  portrays  this  tradeoff. 


Fig.  1.  Tradeoff  analysis  [1]. 

Realistically,  the  nature  of  projecting  data  for  a 
warfare  system  platform  that  has  a  30  year  life  cycle 
creates  a  significant  variance  on  the  data  from  any 
estimate.  In  the  words  of  Paul  G.  Hough,  one  of  RAND 


Corporation's  premiere  cost  analysts:  "cost  analysis  is 
subject  to  considerable  uncertainty"  [2], 

The  factors  that  contribute  to  the  inexactness  of  cost 
estimates  range  from  forecasting  the  cost/performance 
trends  for  new  technology  to  projecting  the  rate  of 
inflation  for  the  life  of  the  system.  Short  term  estimates 
retain  more  accuracy  than  long  term  estimates;  however, 
military  systems  require  long  term  estimates.  For  these 
reasons,  the  tradeoff  analysis  often  relies  on  qualitative 
data  rather  than  quantitative  data.  The  graph  in  figure  2 
illustrates  the  difficulty  historically  encountered  when 
forecasting  the  life  cycle  cost  for  complex  military 
systems. 


Probability 
that  actual 
cost  exceeds 
estimated  cost 
amount  shown 
by  no  more  than 
(percent) 


Development  Plus  Production  Costs 


Error  In  Cost  Estimate  (Percent) 


Fig.  2.  Cost  estimation  track  record  [3]. 

This  paper  describes  an  approach  for  integrating  cost 
modeling  capability  with  the  systems  engineering  tools 
developed  by  the  Engineering  of  Complex  Systems 
Block  Program.  The  approach  will; 

•  produce  qualitative  cost  trends, 

•  incorporate  detailed  short  term  cost  models, 

•  use  heuristics  for  long  term  cost  estimates,  and 
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•  incorporate  existing  n.odels  and  tools,  rather  than 

develop  new  ones. 

The  goal  is  to  enable  systems  engineers  to  perform 
qualitative  cost  tradeoffs  while  synthesizing  complex 
engineering  systems.  Once  the  qualitative  cost 
estimation  capability  has  been  incorporated  into  a 
systems  engineering  tool  set,  additional  research  can 
expand  and  refine  the  capability  to  provide  more  accurate 
cost  estimation  capability. 

The  discussion  about  the  cost  modeling  integration 
approach  advocated  within  this  paper  includes:  a 
historical  overview  of  cost  modeling  within  the 
Department  of  Defense,  a  review  of  critical  system  design 
factors  (SDFs)  related  to  life  cycle  cost,  an  assessment  of 
existing  cost  modeling  tools,  and  a  methodology  for 
integrating  cost  models  with  the  systems  engineering 
tools  developed  under  the  Engineering  of  Complex 
Systems  (ECS)  Block  Program.  System  Design  Factors 
identify,  and  quantify  important  aspects  of  a  system 
design  [4].  The  paper  includes  an  extensive 
bibliography  for  the  reader's  benefit. 

II.  Historical  Overview 

From  its  inception  as  a  specialized  field  in  the  early 
1950's  to  the  present  time,  military  cost  analysis  has 
evolved  into  an  integral  component  of  the  decision 
making  process  for  military  system  development.  As 
cost  analysis  evolved,  new  methodologies  were 
developed  to  improve  the  estimates. 

In  the  1950's  cost  analysis  primarily  evaluated  cost 
estimates  provided  by  contractors.  Analysts  compared 
the  estimates  against  the  cost  of  similar  systems.  This 
estimation  methodology  became  known  as  "analogy"  and 
will  be  described  later  in  this  section. 

The  1960's  was  characterized  by  centralized  decision 
making  brought  about  by  Secretary  of  Defense  Robert 
McNamara  and  his  "whiz  kids".  Cost  analysis  became  an 
important  aspect  of  the  budgeting  process  as  the  nation 
struggled  with  funding  the  Vietnam  War  effort  without 
adversely  affecting  the  economy  [2]. 

In  the  1970's  cost  became  a  parameter  of  equal 
importance  to  performance.  DoD  Directive  5000.1 
instituted  the  Design  to  Cost  (DTC)  concept.  DTC  led 
to  the  development  of  parametric  model  cost  estimating. 
Life  cycle  costing  (LCC)  emerged  when  it  became 
apparent  that  the  existing  models  were  deficient  in 
estimating  Operating  and  Maintenance  (O&M)  costs  (2], 

In  the  1980's  the  Reagan  administration  instituted 
the  Carlucci  initiatives.  The  initiatives  set  out  to 
improve  the  military  procurement  process.  The  legacy  of 
the  1980's  increased  the  media  and  public  focus  on  cost 


analysis  due  to  congressional  hearings,  legislation  and 
high  profile  weapons  systems. 

Currently  in  the  1990’s  DoD  has  focused  on 
developing  affordable  military  systems.  Cost  has  become 
a  driving  parameter.  As  a  result  the  accuracy  of  cost 
estimates  is  critical;  however,  current  state-of-the-art  cost 
analysis  does  not,  in  most  cases,  provide  better  than  a 
rough  order  of  magnitude  estimates  (7).  Consequently, 
tradeoff  analyses  often  rely  on  qualitative  rather  than 
quantitative  estimates. 

The  importance  of  LCC,  identified  in  the  70's,  is 
reflected  in  the  concept  of  viewing  the  life  cycle  and  the 
associated  costs  in  three  distinct  phases.  These  phases  are 
research  and  development  (R&D),  investment  (including 
production)  and  operation  and  maintenance  (O&M). 

R&D  cost  refers  to  all  costs  associated  with 
research,  development,  test  and  evaluation  of  the  system. 
This  covers  all  costs  during  the  validation  and  full  scale 
development  phase  of  a  program.  The  costs  associated 
with  this  phase  end  with  the  satisfactory  completion  of 
the  initial  operational  test  and  government  approval  for 
use. 

Investment  cost  refers  to  all  costs  associated  with 
the  production  of  the  system.  The  costs  incurred  during 
this  phase  are  complete  when  the  operational  system  is 
delivered  to  the  procuring  command  for  use. 

O&M  cost  refers  to  all  cost  associated  with  the 
operation  and  logistics  support  of  the  system  subsequent 
to  the  delivery  of  the  system. 

DoD  cost  analysts  focus  on  these  three  phases  of  the 
life  cycle  and  their  contribution  to  the  overall  life  cycle 
cost.  Cost  related  system  design  factors  identify  the 
components  associated  with  each  phase  in  greater  detail. 

HI.  System  Design  Factors 

System  Design  Factors  provide  a  mechanism  to 
quantify  system  characteristics  for  tradeoff  analyses  [4]. 
Tables  I-III  contain  the  SDFs  for  cost  and  their 
assignment  to  the  three  phases  of  the  life  cycle. 

When  considering  SDFs  related  to  cost,  a  systems 
engineer  should  concentrate  on  the  most  critical  SDFs 
for  each  stage  of  the  life  cycle.  Focusing  on  the  SDFs 
that  provide  the  most  significant  contribution  to  the  cost 
at  each  stage  of  the  life  cycle,  reduces  the  number  of 
factors  required  for  a  total  rough  order  of  magnitude  cost 
estimate. 

For  R&D  cost  the  significant  SDFs  are: 
engineering,  software,  documentation,  and  test  and 
evaluation.  Engineering  includes  costs  related  to 
systems  engineering  and  integration,  design  engineering, 
design  support,  and  the  redesign  or  formulation  of 
engineering  changes  [5]. 
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TABLE  I 

R&D  SYSTEM  DESIGN  FACTORS  [5] 


R&D 

Validation 

Engineering 

5  e 

Program 

Management 

Prototype 

Hardware 

.  i  &  Test 
Equipment 

Documentation 

Test  Spares 

Test  Facilities 

Test  Personnel 
Training 

Test  & 

Evaluation 

TABLE B 

INVESTMENT  SYSTEM  DESIGN  FACTORS  [5] 


Investment 

Production 

Hardware 

Program 

Management 

Training 

Integration  & 

Test 

Production  Test 
and  Evaluation 

Documentation 

Production 
Support  & 
Services 

Industrial 

Facilities 

Initial  Spares 

Installation  and 
Checkout 

■  i/t  iM 

Transportation 

Operational  Sites 

Maintenance 

Sites 

Supply 

Introduction 

TABLE m 

O&M  SYSTEM  DESIGN  FACTORS  [5] 


O&M 

Crew 

Facilities 

Material 

Personnel 

Modernization 

Overhaul 

Software 

Maintenance 

Preventative 

Maintenance 

Corrective 

Maintenance 

Operator 

Training 

Support  and  Test 

Equipment 

Maintenance 

Replenishment 

Spares 

Inventory 

Storage 

Supply  System 
Management 

Repair  Material 

Documentation 

Maintenance 

Shop  Space 

Shipping 

Packaging 

Material 

Material 

Handling  Labor 

For  Investment  cost  (includes  production)  the 
significant  SDFs  are:  production  hardware,  production 


support  and  services,  initial  spares,  operational  sites,  and 
maintenance  sites. 

For  Operating  and  Maintenance  cost  the  significant 
SDFs  are:  crew,  material,  preventative  maintenance, 
corrective  maintenance,  and  modernization. 

The  aforementioned  cost  related  system  design 
factors  become  the  focus  of  cost  estimation  efforts  for  a 
warfare  system.  Analysts  have  developed  several 
techniques  to  estimate  the  components  of  life  cycle  cost 
for  military  systems. 

IV.  Cost  Model  assessment 

Presently  DoD  cost  analysts  use  four  basic  cost 
estimation  methods  [6],  These  methods  are:  analogy, 
expert  judgment,  bottom  up  (Industrial  Engineering),  and 
top  down  or  parametric  estimation.  Table  IV  presents  a 
comparison  of  the  four  methods. 


TABLE  IV 

METHODOLOGY  COMPARISON 


Method 

Accuracy 

Time 

Historical 

Dependence 

Analogy 

Low 

Medium 

High 

Expert 

Judgment 

Low 

Low 

High 

Bottom  Up 

Medium 

High 

Medium 

Top  Down 

Medium 

Low 

Low 

Analogy  involves  the  comparison  of  similar  systems. 

The  analyst  compares  attributes  of  similar  systems  to 
determine  a  reasonable  cost  estimate.  This  approach 
requires  a  fair  amount  of  experience,  as  well  as  historical 
data.  The  estimate  is  highly  subjective  to  the  bias  and 
experience  of  the  analyst.  The  method  assumes  the  use 
of  similar  technologies.  Nevertheless,  the  estimate  can  be 
generated  quickly. 

Expert  Judgment  is  similar  to  analogy  because  the 
estimate  can  be  generated  quickly.  In  this  approach,  an 
"expert"  uses  his  experience  to  generate  a  cost  estimate. 

As  with  analogy,  the  estimate  is  sensitive  to  inaccuracies 
due  to  the  subjective  nature  of  the  expert’s  opinion.  This 
method  depends  highly  on  the  availability  and  skill  of 
experts. 

The  Bottom  Up  or  Industrial  Engineering  method 
represents  a  more  objective  approach.  This  method 
divides  the  estimation  task  into  smaller  units.  An 
individual  or  group  who  has  the  appropriate  data, 
experience  and  model  to  generate  an  accurate  result 
produces  the  estimate.  The  sum  of  the  unit  costs 
determines  the  total  cost.  This  approach  yields  a  more  '  oa 


3 


□  □53 


Complex  Systems  Engineering  Synthesis  and  Assessment  Technology  Workshop,  July  19-20,  1994 


accurate  estimate,  but  it  is  time  consuming.  The 
availability  of  unit  cost  data  determines  the  accuracy  of 
the  Bottom  Up  approach. 

Top  Down  or  Parametric  Estimation  is  the  most 
widely  used  cost  estimation  method  within  DoD  [6], 
This  method  relies  on  computer  models  to  obtain  the 
cost  estimate.  The  analyst  provides  lower  level  design 
parameters  as  input  to  the  model.  Generally  these  models 
are  easy  to  use,  and  produce  a  more  accurate  result.  The 
difficulty  with  using  parametric  models  is  that 
occasionally  the  required  input  parameters  may  not  be 
known  or  may  be  difficult  to  quantify.  The  lower  level 
nature  of  some  of  the  parameters  often  means  they  are 
difficult  to  define  until  the  design  progresses. 

DoD  uses  parametric  models  primarily  to  estimate 
software  costs.  One  reason  for  this  is  that  software  cost 
is  a  significant  component  of  the  system  life  cycle  cost. 
In  addition  it  appears  that  software  cost  may  be  more 
difficult  to  determine  using  the  other  estimation 
methodologies. 

A  variety  of  software  cost  estimation  models  exist. 
Many  are  proprietary.  The  more  widely  used  models 
include  [6]: 

•  Constructive  Cost  Model  (COCOMO), 

•  PRICES, 

•  System  Evaluation  and  Estimation  of  Resources 
(SEER), 

•  Software  Life  Cycle  Cost  Model  (SLIM), 

•  Revised  Enhanced  Version  of  Intermediate 
COCOMO  (RE VIC),  and 

•  Software  Architecture,  Sizing  and  Estimating  Tool 
(SASET). 

The  models  require  the  user  to  provide  specific 
parameters  that  quantify  and  describe  their  system.  The 
parameters  required  by  the  models  cover:  lines  of  code, 
language,  number  of  programmers,  experience  of 
programmers,  development  platform,  availability  of  tools, 
reliability  and  schedule.  Analysts  may  have  difficulty 
quantifying  some  parameters  requested,  such  as  developer 
experience.  However,  these  models  provide  an  estimate 
rather  quickly.  In  addition,  the  models  are  generic,  that 
is  the  software  models  can  be  applied  to  estimate  costs 
for  any  application.  System  engineers  using  these 
models  should  understand  that  these  models  often  are 
difficult  to  calibrate  [7],  and  require  recalibration  for 
different  environments  [8]. 


While  many  generic  models  exist  for  software  cost 
estimation,  the  same  is  not  true  for  other  aspects  of  the 
life  cycle  cost.  These  costs  are  generally  computed  by 
analogy,  expert  opinion  or  by  models  developed  for  a 
specific  application.  The  models  use  heuristic  data  bases 
from  existing  combat  systems  to  estimate  cost.  For 
example  at  the  Naval  Undersea  Warfare  Center  (NUWC), 
several  combat  system  specific  cost  estimation  models 
exist  based  on  data  from  existing  submarine  warfare 
systems. 

The  current  state-of-the-art  in  cost  estimation 
technology  relies  on  generic  models  to  estimate  software 
costs  and  expert  judgment/heuristic  models  for  other 
components  of  the  life  cycle  cost.  Therefore,  the 
integration  of  cost  models  with  systems  engineering 
tools  should  be  flexible  to  incorporate  the  different  types 
of  models  available  for  each  phase  of  the  life  cycle. 

V.  INTEGRATING  COST  MODELS 

The  integration  of  cost  modeling  capability  into 
systems  engineering  tools  will  enable  systems  engineers 
to  perform  qualitative  cost  tradeoffs  while  synthesizing 
complex  engineering  systems.  The  objectives  of  the  cost 
model  integration  process  encompass: 

•  defining  cost  related  systems  design  factors, 

•  pinpointing  critical  cost  related  system  design 
factors, 

•  providing  systems  engineers  with  qualitative  cost 
factors  for  systems  design  synthesis, 

•  linking  cost  related  system  design  factors  to  the 
Design  Structuring  and  Allocation  Optimization 
(DESTINATION)  systems  engineering  tool,  and 

•  permitting  design  factor  optimization  within 
DESTINATION. 

Existing  cost  models  will  be  used  in  order  to 
expedite  the  availability  of  cost  modeling  capability. 
Figure  3  depicts  the  method  which  will  integrate  cost 
models  with  the  DESTINATION  systems  engineering 
tool  developed  by  the  Engineering  of  Complex  Systems 
Program. 

The  method  breaks  down  the  cost  models  into  three 
different  categories:  hardware  research  and  development, 
software  research  and  development,  and  the  remaining  life 
cycle  costs.  The  hardware  and  software  research  and 
development  cost  models  are  kept  separate  because 
detailed  cost  databases  exist  for  hardware,  and 
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established  models  exist  for  software  (refer  to  the  Cost 
Model  Assessment  section  of  this  paper  for  examples  of 
existing  databases  and  models).  Heuristics  will  provide 
information  for  the  remaining  part  of  a  system's  life 
cycle.  Heuristics  are  readily  available  and  produce 
satisfactory  information  for  this  stage  of  the  cost  model 
integration  process. 

The  link  to  DESTINATION  will  occur  through  the 
Systems  Engineering  Technology  Interface  Specification 
(SETIS)  software.  SETIS  interface  software  will  be 
developed  to  provide  a  link  to  the  cost  models.  The 
SETIS  link  ensures  compatibility  with  other  tools 
developed  within  the  Engineering  of  Complex  Systems 
Program.  System  design  factors  describing  the  hardware 
resources,  software  design,  functional  requirements  and 
programmatic  requirements  will  be  extracted  from 
DESTINATION  through  SETIS.  Additional,  lower 
level,  SDFs  will  be  defined  during  the  model  integration 
process.  The  cost  model  will  supply  information 
regarding  cost  related  system  design  factors  to 
DESTINATION  through  SETIS. 


Fig.  3.  Integration  with  DESTINATION. 

If  the  system  specific  information  currently  provided 
by  DESTINATION  is  inadequate,  then  the  user  must 
provide  additional  information  to  the  cost  models.  The 
SETIS  interface  will  also  query  the  user  to  input  the 
additional  information  and  incorporate  the  data  into  the 
cost  estimate. 

The  SETIS  interface  will  manage  the  data  generated 
from  the  hardware  R&D,  software  R&D,  and  life  cycle 
cost  models.  The  interface  software  will  require  the 


development  of  C++  code  to  maintain  compatibility  with 
SETIS.  SETIS  was  implemented  using  a  C++  class 
hierarchy  [9].  The  intent  is  to  develop  cost  modeling 
capability  that  requires  minimum  modification  to 
existing  SETIS  software. 

The  cost  related  system  design  factors  described  in 
this  paper  will  serve  as  the  foundation  for  the  cost 
models.  The  system  design  factors  listed  in  NAVS WC 
Technical  Report  92-268  will  also  receive  consideration. 
These  system  design  factors  include  cost  to:  develop, 
prototype,  produce,  test,  purchase,  operate,  maintain, 
repair,  include  security  capability,  and  achieve 
productivity. 

Once  the  concept  has  been  demonstrated,  the 
repertoire  of  cost  models  can  be  expanded  to  give  the 
systems  engineer  different  options  and  perspectives. 
Each  new  model  would  be  linked  to  DESTINATION 
through  SETIS,  and  the  SETIS  interface  software 
developed  for  the  cost  models. 

In  addition  to  the  integration  of  cost  modeling 
capability  with  DESTINATION,  the  approach  outlined 
can  also  be  applied  to  the  integration  of  commercially 
available  system  engineering  tools.  By  using  the  CASE 
Document  Interchange  Format  (CDIF)  instead  of  SETIS, 
the  cost  estimation  models  can  be  integrated  with  system 
engineering  tools  such  as  RDD-100,  Cadre  Teamwork 
and  others.  Of  course  this  approach  assumes  that  these 
tools  comply  to  the  standardization  of  CDIF.  If  the 
selected  tools  don't  comply  with  CDIF,  some  translation 
between  specific  formats  may  be  required  to  interchange 
data  between  tools. 

VI.  SUMMARY 

This  paper  describes  an  approach  for  integrating  cost 
modeling  capability  with  the  systems  engineering  tools 
developed  by  the  Engineering  of  Complex  Systems 
Block  Program.  The  approach  entails: 

•  using  existing  hardware  R&D  cost  databases, 

•  incorporating  established  software  cost  models, 

•  estimating  long  term  life  cycle  cost  with  heuristics, 

•  defining  cost  related  system  design  factors,  and 

•  designing  a  software  interface  to  DESTINATION 

via  SETIS. 

The  cost  modeling  integration  approach  was 
conceived  by  tracing  through  the  history  of  cost 
modeling  within  the  Department  of  Defense,  defining 
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cost  related  system  design  factors,  assessing  existing  cost 
modeling  tools,  and  formulating  a  methodology  for 
integrating  cost  models  with  the  systems  engineering 
tools  developed  under  the  Engineering  of  Complex 
Systems  Block  Program. 

A  qualitative  cost  estimation  capability  will  enable 
system  engineers  to  perform  tradeoffs  while  synthesizing 
complex  engineering  systems.  The  fundamental  tradeoff 
between  development  cost,  long  term  life  cycle  cost,  time 
and  performance  is  the  primary  objective. 

Finally,  the  cost  modeling  research  that  led  to  the 
proposed  cost  modeling  approach  produced  an  extensive 
list  of  references.  Although  many  of  these  references 
were  not  explicitly  used  within  the  context  of  this  paper, 
the  full  list  is  included  for  the  reader's  benefit. 
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